M.i.T.  LIBRARfES  -  DEWEY 


Q> 


HD28 

.M414 


'^ 


Dewey 


Integrating  the  Choice  of  Product  Configuration 
with  Parameter  Design 

M.  de  Falco  and  K.  T.  Ulrich 

WP  #3749-94  December  1994 


;/iAsaACHijstns  iNSvirufe 

OF  TECHN'OLOGY 

JAN  19  1995 

LIBRARIES 


Integrating  the  Choice  of  Product  Configuration 
with  Parameter  Design 


Massimo  de  Falco 

University  of  Naples  "Federico  11" 

Dpt.  of  Materials  and  Production  Engineering 

P.le  Tecchio,  Naples   80125  -  Italy 

Karl  T.  Ulrich 

Massachusetts  Institute  of  Technology 

Sloan  School  of  Management 

Cambridge,  MA  02139  -  USA 


December  1994 


Abstract 

Parameter  design  and  experimental  techniques  to  develop  new  products 
essentially  focus  on  the  optimization  of  a  single  feasible  configuration  of  a 
product.  This  paper,  using  an  example,  illustrates  how  to  extend  factor 
optimization  to  system  design  to  support  the  designer  in  the  choice  of  the 
product  configuration. 

The  proposed  approach  integrates  the  design  phases  to  reach  the  best 
configuration  using  a  process  aimed  at  subsequent  improvement  of  the 
product,  providing  confidence  in  the  information  underlying  the  decision 
making  process,  and  correctly  testing  all  hypotheses  and  assumptions. 
In  this  way,  this  paper  is  a  further  attempt  to  overcome  barriers  in  industrial 
practice,  by  combining  the  engineering  knowledge  with  effective  statistical 
techniques. 
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1   Introduction 

The  current  concept  of  quality  indicates  that  quality  must  be  "designed  in"  the 
product,  and  that  other  forms  of  interventions  present  limitations  and  may 
not  be  cost  effective  [Unal  R.,  1993].  In  typical  product  development  practice 
"Quality  by  Design"  consists  of  a  three-step  approach:  system  design,  applies 
the  available  knowledge  to  produce  a  basic  functional  prototype;  parameter 
design,  identifies  the  setting  of  parameters  which  optimize  the  system 
performance;  and  tolerance  design,  establishes  limits  on  allowable  variation 
in  the  parameters  [Dehnad  K.,  1988].  However,  this  methodology  is  only  a 
partial  upstream  extension  of  design  for  quality,  since  it  works  only  on  a 
predefined  product  concept,  and  thus  does  not  include  the  optimization  of 
the  product  configuration. 

Using  the  Powered  Fruit  Juicer  example,  in  this  work,  we  will  show  how 
experimental  design  can  be  used  to  evaluate  complex  architectural  factors 
such  as  different  design  solutions  for  head  shape,  or  various  technologies 
such  as  two  systems  of  head  rotation,  and  how  these  choices  affect  final 
performance.  Including  system  design  in  the  experimental  optimization,  this 
approach  moves  the  development  process  naturally  toward  greater 
integration. 

The  structure  of  this  paper  reflects  the  temporal  differentiation  in  the 
underlying  methodology  between  the  a  priori  stage,  in  which  the  "product 
framework"  and  the  "experiment  pattern"  are  defined  and  the  a  posteriori 
stage  in  which,  after  experimentation,  analysis  and  feedback  is  conducted  to 
complete  the  development  cycle. 


1.1  Current  Problems  with  System  Design 

The  typical  new  product  development  approach  in  industry  has  been 
characterized  by  strong  pressure  to  reduce  time  and  improve  quality,  while 
the  lack  of  planning  and  statistical  tools  are  still  strongly  felt  in  all  phases  of 
design  [Goh  T.N.,  1991].  One  of  the  consequences  of  this  development 
approach  is  a  process  where  efforts  are  concentrated  on  the  realization  of  the 
first  prototype  and  its  subsequent  optimization  through  the  definition  of 
tolerances  [Unal  R.,  1993]. 

Pushing  to  reach  quickly  a  first  feasible  product  architecture  with  subsequent 
row  tuning  of  parameters  under  cost  and  quality  constraints  has  already  been 
demonstrated  to  be  ineffective.  As  Taguchi  pointed  out  a  long  time  ago,  to 
make  products  with  high  performance  and  robustness,  a  design  process  to 
optimize  all  key  factors  is  necessary.  This  phase,  known  as  parameter  design, 
identifies  the  product's  parameter  settings  that  reduce  the  sensitivity  to 
sources  of  variation.  Even  if  this  approach  is  essentially  the  classical  Design 
of  Experiment  [Kacker  R.N.,  1991],  Taguchi  popularized  this  approach  in 
industrial  applications  [Pignatiello  J. J.,  1992]. 

Concentrating  efforts  on  the  first  selected  product  solution,  in  a  strongly 
sequential  way  that  does  not  allow  either  for  subsequent  iteration  or  ongoing 
feedback  represent  a  weakness  in  this  traditional  approach.  This  tendency 
endures  in  spite  of  the  possibility  offered  by  the  prototype  as  a  milestone  to 
close  the  loop  in  the  early  stages  of  product  design,  and  of  all  verifications 
required  to   validate  the  assumptions  of  the  statistical  analysis. 

1.2  Objectives  of  Proposed  Approach 

The  present  work  points  out  the  importance  of  and  the  possibility  of 
extending  factor  optimization  back  to  system  design  where  the  architecture  of 
the  product  is  not  yet  frozen  (Exhibit  1).  This  main  objective  is  also  combined 
with  the  integration  of  design  phases  to: 

-  predispose  and  activate  a  continuous  improvement  process, 

-  test  correctly  all  hypotheses  and  assumptions, 

-  know  the  reliability  and  confidence  of  the  information  underlying  the 
decision-making  process. 


Exhibit  1.  Extension  of  the  approach  in  the  development  process- 
Extended  Approach 


Idea-Need  Analysis 
Features  Identification 
Cause-Effect  Analysis 
Feasibility  Analysis 


Architecture  Definition 
Solutions  Compilation 
Alerna'tives'  Comparison 
Architecture  Selection 


Optimization 
Parameter  Setting 
Variation  Reduction 
Tolerance  Setting 


These  goals  are  intended  to  achieve  in  the  design  process  flexibility  and 
control.  The  flexibility  is  relevant  because  it  avoids  a  situation  in  which,  after 
the  prototyping  of  product,  the  configuration  is  treated  as  fixed.  In  fact,  the 
early  freezing  of  the  architecture  has  a  strong  impact  either  in  finding  the  best 
overall  solution  for  the  product  or  in  the  creation  of  knowledge  about  the 
other  alternatives.  This,  integrated  with  the  feedback,  becomes  one  of  the 
keys  to  move  the  process  toward  the  optimum,  to  formalize  the  knowledge 
and  to  avoid  common  mistakes  occurring  in  the  early  stage  of  the  design 
phases. 


2  Approach 

2.1  The  "Powered  Fruit  Juicer"  Example 

An  application  of  the  proposed  approach  has  been  conducted  on  a  consumer 
product,  the  Powered  Fruit  Juicer  (PFJ),  using  brands  already  on  the  market  as 
prototypes  for  the  development  of  a  potential  next  generation.  This  example 
will  be  used  to  illustrate  the  different  steps  of  the  present  work,  referring,  in 
cases  where  the  example  does  not  apply,  to  hypothetical  situations. 
The  Powered  Fruit  Juicers  are  home  products  mostly  used  for  oranges,  and  all 
manufacturers  offer  products  with  similar  designs.  The  juicers  essentially 
work  through  head  rotation  while  the  user  holds  the  fruit  and  applies 
pressure.  These  products  exhibit  some  interesting  features  that  are  perhaps 


driven  by  the  developer's  experience  and  by  n:\arketing  considerations  rather 
than  by  objective  performance  evaluation.  For  this  reason  this  example 
appears  to  be  appropriate  for  an  application  of  the  proposed  approach,  which 
introduces  architectural  performance  evaluation  as  part  of  the  product  design 
process. 

2.2  Logic  and  Stages  of  the  Approach 

In  our  approach,  two  main  concepts  differentiate  partially  the  design  process 
from  the  traditional  methodology.  These  are:  the  "framework  of  the  product" 
and  the  "pattern  of  the  experiment"  (Exhibit  2).  The  first  relates  to  the 
product/process  under  evaluation  and  synthesizes  present  knowledge  about 
it.  The  second  refers  to  the  planned  experimental  process,  respecting 
constraints  and  objectives,  to  improve  the  level  of  knowledge.  The  approach 
also  identifies  two  stages  of  the  improvement  cycle,  a  priori  and  a  posteriori 
of  the  experiment. 

With  only  one  iteration,  like  in  the  PFJ  case,  the  design  process  provides  the 
first  best  configuration  of  the  product  and  updates  related  know-how. 


Exhibi 

2.  Main  steps  of  the  approach. 
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2.2.1  Framework  of  the  Product 

The  concept  of  framework  is  an  upstream  extension  of  the  product 
architecture.  In  fact,  it  includes  the  final  architecture  and  other  feasible 
alternatives,  and  is  expressly  defined  to  apply  during  the  experimental  phase. 
In  this  way  the  framework  constitutes  the  experiment  plan  relative  to  the 
product  and,  although  interrelated  with  the  pattern  of  experimentation,  it 
helps  to  simplify  and  to  clarify  the  process. 

In  order  to  settle  the  product  framework,  as  illustrated  in  the  a  priori  stage  of 
Exhibit  2  ,  it  is  necessary  to  determine  the  predominant  performance  of  the 
product,  the  hypotheses  to  investigate  and  the  feedback  plan. 

The  performance  under  evaluation,  and  the  response  factor  associated.  This 
is  a  delicate  point  because,  through  a  cause-effect  analysis,  the  performance 
will  be  linked  to  the  key  factors,  and  the  response  factor  can  affect  either 
controllable  or  uncontrollable  factors.  In  the  PFJ  case  the  performance  under 
evaluation  was  the  juice  yield,  and  the  response  factor  was  the  ratio  between 
the  juice  produced  and  the  juice  that  remained  in  the  fruit.  The  characteristic 
of  this  response  factor  is  its  very  low  sensitivity  to  the  "juiciness"  of  a  specific 
fruit,  a  critical  element  in  the  experimentation,  but  it  remains  somehow 
sensitive  to  the  dimension  and  the  shape  of  the  fruit. 
Exhibit   3.  System  table. 
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ArchKecture  factor  =  factor  emlxxJied  in  the  prototypes; 
Steady  factor  =  factor  settled  for  at)  the  experiment; 

Minimized  factor     =  factor  with  reduced  variability; 


Setting  factor  =  factor  varied  In  the  experiment; 

Variable  factor  =  factor  not  oontmlled; 

Uncontrolled  factor  =  factor  uncontrollable. 


The  principal  hypotheses  to  investigate.    These  synthesize  the  engineering 
and  scientific  knowledge  about  the  phenomenon  under  evaluation.    They  are: 

•  the  classification  of  the  factors  playing  significant  roles,  with  the  choice  of 

selected  factors,  which  are  considered  responsible  for  the  main  effects  on  the 
performance.  Exhibit  3  shows  the  different  categories  using  factors 
identified  for  the  PFJ  application  with  oranges. 

The  selected  factors,  which  will  be  directly  observed,  are  divided  into 
architecture  factors  and  setting  factors;  the  first  group  constitutes  the 
peculiarities  of  the  product  concept  or  configuration.  In  the  case  of  PFJ,  this 
is  composed  of  two  design  factors,  (1)  the  head  shape  (Exhibit  4)  and  (2)  the 
power,  applied  with  different  electric  motors  to  the  heads;  then  a 
technological  factor,  the  head  rotation  scheme,  where  the  first  type  is  a  one- 
way standard  rotation  and  the  second  type  is  an  alternated  rotation  in  two 
directions  (Exhibit  5).  The  architecture  factors,  as  distinct  from  the  setting 
factors,  require  more  prototypes  or  a  modular  prototype  with  the  possibility 
to  exchange  the  parts  during  the  experiment,  and  constitute  the  critical 
elements  to  investigate. 

Exhibit  4.  The  two  different  solutions  of  head  shape. 


Design  Factor 

Head  Shapes 


Type  I 


Type  I  I 


It  is  worth  mentioning  that  not  all  categories  identified  for  unselected  and 
uncontrollable  factors  have  to  be  present,  but  all  of  them  influence  the 
experimental  results  and  are  helpful  for  correctly  planning  the  experiment. 
The  choice  of  juice  yield  as  response  factor  implies  the  minimization  of 
variation  in  the  orange  dimension  to  reduce  the  uncontrolled  variability  in 


the  experiment;  while  the  shape  of  the  orange  was  left  uncontrolled  because 
of  technical  difficulties  in  measurenient. 


Exhibit  5.  The  two  different  technologies  applicable  to  the  PFJ. 
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•  The  interactions  among  the  factors  selected.  In  table  6  is  represented  an 
interaction  diagram  showing  the  relationships  among  the  factors  and  their 
intensities.  These  connections  are  generally  based  on  previous  experiences 
and  reflect  present  knowledge. 

Exhibit  6.   Interactions  diagram,  the  a  priori  hypotheses  on  interactions. 


=  strong  Effect 
=  Weak  Effect 


•  Scaling  and  leveling  systems.  Assuming  a  metric  and  defining  the  levels  at 
which  the  factors  have  to  be  observed,  we  can  face  factors  that  are  not 
measurable  and/or  not  continuous.  In  these  cases,  very  frequent  with 
architecture  factors,  it  is  necessary  to  divide  the  factors  into  different  types 
(Table  1).  For  continuous  factors,  on  the  other  hand,  we  must  determine 
their  levels.  In  the  PFJ  case  the  "range"  was  defined  as  the  ratio  between  the 
difference  of  two  levels  and  their  middle  point;  this  range  should  be 
significantly  close  to  the  percentage  of  the  known  or  supposed  variation  of 
the  factors.  Erroneous  definitions  of  levels  can  lead  to  altered  conclusions, 
as  we  will  see  for  the  time  factor. 


Table  1.  Leveling  for  the  factors  c 

f  PFJ  case. 

Letter 

Factor 

Level  1 

Level  2 

Range 

A 

head  shape 

Type  I 

Type  II 

. 

B 

power 

18  watt 

33  watt 

67% 

C 

rotation 

Oneway 

Two  ways 

- 

Dl 

pressure 

0.12Kg/cm2 

0.08  Kg/cm2 

40% 

D2 

time 

10  sec 

16  sec 

46% 

D3 

slope 

0° 

20° 

~ 

Feedback  planning.  Since  one  relevant  part  of  the  system  design  is  the 
application  of  scientific  and  engineering  knowledge  to  develop  feasible 
configurations  of  the  product,  which  is  based  on  human  intuition  and 
experience,  it  is  helpful  to  impose  some  structure  to  this  process  [de  Falco  M., 
1993].  In  fact,  the  improving-through-learning  process  [Deming  W.E.,  1986] 
requires  that  all  information  is  synthesized  and  recorded  in  a  form  that 
allows  subsequent  comparison  with  new  information,  and  makes  this  data 
accessible  to  those  who  can  intervene  in  the  design  in  its  early  stages  [Box  G., 
1988].  Examples  of  possible  formalizations  are  the  system  table  and  the 
interactions  diagram  developed  in  the  PFJ  example.  Analogous  work  can  be 
done  with  the  well  known  cause-effect  diagram. 

2.2.2  Pattern  of  Experiment 

The  experimental  phase  is  the  stage  where  we  try  to  understand  better  a 
phenomenon  of  the  real  world  through  experimentation  [Winer  B.J.,  1970]. 
Since  there  always  is  a  tradeoff  between  cost  and  the  usefulness  of  data,  it  is 


important  to  plan  the  experinnent  in  order  to  achieve  the  right  data  at  the 
lowest  cost.  This  is  particularly  true  in  the  prototyping  phase,  where  we  are 
often  restricted  by  several  technical  and  economical  constraints. 

The  central  point  in  this  tradeoff  is  the  number  of  trials  to  run  during  the 
experiment,  which  is  linked  with  the  number  of  factors  and  interactions  we 
can  investigate,  and  the  precision  of  the  data  collected.  Once  this  boundary  is 
defined,  there  still  are  many  parameters  and  structures  to  define  in  order  to 
conduct  the  experiment  that  will  yield  the  desired  outcome.  All  these 
parameters  along  with  the  structure  choices  determine  the  "pattern  of 
experiment". 

The  outcome  of  the  experiment,,  although  unknown,  is  completely  defined 
once  we  have  the  framework  of  the  product  under  study  and  the  definition  of 
the  pattern  of  experiment  in  terms  of  design  constraints  and  leverages. 

Design  Constraints 

•  Physical  constraints.  Once  all  factors  are  characterized  and  the  factors  to 
control  during  the  experiment  are  selected,  it  is  essential  to  understand  if 
some  restrictions  exist  in  the  measurement  of  these  due  to  the 
characteristics  of  the  prototypes. 

Some  of  the  typical  restrictions  that  can  occur  are  the  "nested  factors" 
and /or  "split-plot  factors".  These  are  particularly  frequent  in  prototype 
testing  because  architecture  factors  embodied  in  the  prototypes  are  often  not 
separable.  In  fact,  each  prototype  can  constitute  a  specific  solution  that, 
because  of  the  different  design,  assembly  and  parts,  can  interact  with  other 
factors  generating  altered  effects.  In  the  PFJ  pattern  (Exhibit  7)  we  face  a 
structure  of  prototypes  where  the  power  factor  is  nested  in  the  design  factor. 
Precisely  because  of  the  two  different  electric  motors,  the  output  levels  of 
power  are  similar  but  not  equal,  even  with  the  same  power  input.  An  easy 
way  to  recognize  if  a  factor  (X)  is  nested  in  another  (Y)  is  to  verify  if  each 
level  of  X  is  combined  with  every  levels  of  factor  Y  [Mason  R.L.,  1989; 
Winer  B.J.,  1970]. 

The  main  consequence  of  this  restriction  is  that  the  effect  of  the  power  will 
be  confounded  with  the  design-power  interactions,  and  these  data  will 
require  a  different  treatment  (Appendix  A). 


Exhibit  7.  Pattern  of  the  Powered  Fruit  Juicer  experiment. 
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•  Economical-technical  constraints.  These  constraints  determine  the  number 
of  prototype  variants  we  can  build  and  the  number  of  trials  feasible,  and  so 
constitute  the  upperbound  for  the  total  number  of  runs  in  the  experiment. 

•  Knowledge  constraints.  Since  this  approach  is  structured  as  an  iterative 
process,  the  quality  of  the  pattern,  and  then  of  the  data,  is  linked  to  previous 
knowledge,  of  which  a  fundamental  element  is  the  error  standard  deviation 
of  the  experiment  (Cq).  The  availability  of  this  parameter  allows  a  better 
calculation  of  the  minimum  number  of  trials  that  matches  our  expected 
reliability. 

Design  leverages  are  the  values  to  assign  to  the  free  experiment  parameters  to 
fit  the  strategy  and  design  constraints. 

•  Acceptable  Risk  level.  In  the  statistical  analysis  performed  to  identify  the 
key  factors,  we  can  make  two  different  types  of  error:  Type  I  occurs  when  a 
factor  is  considered  significant  and  in  reality  is  not;  and  Type  11  occurs  when 
a  factor  is  considered  not  significant  and  in  reality  it  is.  We  prevent 
ourselves  from  making  these  errors  by  fixing  the  probabilities  of  their 
occurring  at  a  low  level,  P(Type  I  error)=a  and  P(Type  II  error)=p.  In  the  PFf 
case  we  fixed  a  =0.05  and  P  =0.05,  the  unusually  strict  control  of  the 
parameter  p  is  justified  by  the  importance  of  avoiding  the  rejection  of  a  new 
design  solution  if  it  makes  an  improvement  in  product  performance. 

•  Precision  of  the  experiment.  It  is  the  required  capacity  to  detect  a  specific 
change  in  the  average  of  a  response  factor  due  to  a  factor  or  interaction. 
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This  value  should  be  defined  in  accordance  with  the  variation  considered  of 
interest.  Moreover  we  should  observe  that  risk  level,  precision  and  total 
number  of  trials  are  strongly  interrelated  and  their  choice  n\ust  be  made 
coherently  [Diamond  W.J.,  1981].  In  fact,  a  high  imposed  precision,  without 
changing  other  parameters,  implies  a  high  risk  of  making  type  n  error.  In 
the  PFJ  experiment  the  precision  was  fixed  as  the  capacity  to  detect  a 
variation  of  5=±4%  in  the  ratio  chosen  as  the  response  factor. 

•  Degree  of  freedom  allocation.  Once  the  total  number  of  trials  (N)  with 
previous  parameters  is  defined,  the  total  number  of  degree  of  freedom  (doO 
available  in  the  experiment  (N-1)  is  determined.  However,  there  still  is  the 
possibility  of  deploying  these  dofs  among  factors,  interactions  and  error.  In 
fact,  respecting  constraints  which  require  in  our  case  one  more  dof  for  the 
nested  factor  and  for  each  interaction  with  the  nested  factor,  including  more 
factors  in  the  experiment  means  fewer  dofs  available  for  the  error  and  more 
possibility  to  have  confounded  factors.  This  is  a  reason  why  a  high 
factors /trials  ratio  is  not  indicated  for  high  precision  experiments. 

•  Resolution  of  experiment.  The  last  choice  to  make  is  the  definition  of 
resolution,  with  related  confounding,  for  each  factor  and  interaction  of 
interest  in  the  experiment  [Tsui  K.L.,  1988].  This  choice,  which  completes 
the  pattern  of  experiment,  is  important  because  it  defines  factors  that  will  be 
deeply  understood  and  factors  that  will  remain  somewhat  uncertain  due  to 
confounded  effects. 


Table  2.  Resolution  and  confounding  for  factors  and  interactions  . 

Factor 

Description 

Resolution 

Confounding 

A 

Head  shape 

V 

A.D1.D2.D3 

B 

Power 

m 

A.B  +  B.D1.D2.D3 

C 

Rotation  tech. 

V 

B.D1.D2.D3 

Di 

Pressure 

m 

D2.D3 

D2 

Time 

m 

D1.D3 

D3 

Slope 

m 

D1.D2 

B.C 

Interaction 

V 

B.C.D1.D2.D3 

A.C 

Interaction 

V 

A.C.D1.D2.D3 

A.Dj 

Interactions 

IV 

A.Di.Dk 

B.Dj 

Interactions 

m 

A.B  +  B.Di.Dk 

C.Dj 

Interactions 

IV 

C.Di.Dk 

The  resolution  III  is  obtained  when  a  factor  effect  is  confounded  with  two-factors  interactions,  the 
resolution  IV  when  a  factor  effect  is  confounded  with  three-factors  interactions,  and  so  on.  [Robinson  G.K., 
1993] 
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Any  a  priori  knowledge  about  the  impact  of  some  effects  could  be  a  helpful 
guide  in  the  resolution  decision  [Alkhairy  A.,   1991]. 

Accordingly  with  the  factor  categorization  made  above,  in  the  PFJ  example, 
the  architecture  parameters  were  positioned  in  the  highest  resolution 
position,  because  an  error  in  the  architecture  is  more  serious  than  an  error 
in  the  setting  of  the  parameters  (Table  2).  In  this  way  we  can  perform  few 
trials,  reducing  quality  of  the  data  only  where  eventual  errors  would  be  less 
serious. 

3  The  PFJ  Experiment 

The  experiment  is  completely ,  designed  once  we  have  determined  the 
previous  elements  in  the  framework  and  in  the  pattern.  The  total  number  of 
trials  can  be  calculated  using  tables  available  in  literature  [Davies  D.L.,  1956], 
which  express  the  relationship  /(a,  p,  O,  n)=0  (Appendix  F),  where  0=  8/Ge  is 
the  ratio  between  the  precision  of  the  experiment  and  the  error  standard 
deviation,  n  is  the  sample  size  of  each  group  compared  in  the  experiment 
(equal,  for  two  level  factorial  design,  to  half  of  the  total  number  of  trials). 

In  the  design  of  experiment  of  PFJ,  because  of  a  total  lack  of  knowledge,  we 
conducted  a  preliminary  test  to  estimate  Oq  [Davies  O.L.,  1978],  which  came 
out  to  be  approximately  0.0277.  In  Table  Fl  of  the  Appendix  F,  with  0=1.45 
(=0.04/0.0277),  we  find  n=14.  The  closest  orthogonal  array  is  n=16=N/2  and 
then  the  total  number  of  trials  will  be  N=32.  Another  system  to  determine  O, 
if  Oe  is  not  known,  is  to  define  5  as  percentage  of  Gg.  This  can  be  done  with 
the  help  of  some  conservative  considerations  [Diamond  W.J.,  1981;  Mason 
R.L.,  1989].  The  minimum  number  of  trials  so  calculated  (32)  matches  the 
reduction  imposed  by  the  resolution  choices. 

The  resulting  pattern  of  the  experiment  is  a  full  factorial  for  the  architecture 
factors  and  a  half  factorial  for  the  setting  factors,  thus  we  obtained  the 
resolution  V  for  the  architecture  factors,  a  resolution  III  for  the  setting  factors, 
and  a  resolution  IV  for  the  interactions  between  the  factors  of  the  two 
different  categories. 

The  pattern  is  recognizable  in  Exhibit  8  thanks  to  the  positioning  of  the 
architecture  factor  on  the  top  of  the  matrix,  where  full  factorial  implies  the 
presence  of  all  combinations  among  the  factors.  On  the  side  we  have  the 
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setting  factors  where  the  half  fractional  design  implies  the  presence  of  only 
half  the  data,  but  the  orthogonality  of  the  design  guaranties  the  possibility  of 
comparing  any  factor  at  each  of  its  two  levels. 


Exhibit  8 

Matri 

X  of  experiment  for  the  PFJ  case  with  the  results. 

A 
B 
C 

o 
B 
S 

E 
R 
V 
A 

T. 

D2 

D3 

1  architecture! 

Type  I 

Typen 

la 

2a 

lb 

2b 

1       1       2 

1       1       2 

1       1       2 

1        1       2 

Dl 

1 

1 

1 

0.73 

0.81 

0.65 

0.65 

0.44 

0.62 

0.39 

0.55 

2 

0.94 

0.82 

0.81 

0.91 

0.68 

0.60 

0.57 

0.68 

1 

2 

!5 

- 

2 

0.72 

0.67    1    0.75    1    0.75    |    0.58    |    0.66    |    0.53    |    0.68 

1 

1 

2 

2 

1 

2 

0.55 

0.71    1    0.57    1    0.62    1    0.47    |    0.40    |    0.40    |    0.43 

2 

1 

Assuming  third-order  and  higher  interactions  are  negligible,  the  model 
initially  tested  is: 

Tlijklmn  =  |I  +  tti  +  pj(i)  +  7k  +  5ii  +  52m  +  hn  +  (a7)ik  +  (Py)j(i)k  + 
+  (a5i)ii  +  (a52)im  +  (a53)in  +  (P5i)j(i)l  +  (P52)j(i)m  + 
+  (P53)j(i)n  +  (Y5i)kl  +  (Y52)km  +  (Y53)kn  +  e(ij)klmn 

where,  Greek  letters  represent  the  effects  of  the  respective  factors  indicated 
with  Roman  letters,  the  products  in  parenthesis  are  the  interactions,  |i  is  the 
overall  mean,  j(i)  indicates  that  the  P  effect  is  nested  in  the  a  effect,  and  the 
indexes  i,  j,  k,  1,  m,  n  vary  between  1  and  2  [Montgomery  D.C.,  1984]. 

Therefore,  the  null  and  alternative  hypotheses  to  test  in  the  analysis  were: 

Hq:      Tki  =  Tk2  =  0  ;  HI:      at  least  one  Tki  "^  0  ; 

for  every  k; 

Hq:      (TkTh)ij  =  0;  HI:      at  least  one  (TkTh)ij  ^  0  ; 

for  every  k,  h  and  for  all  i,  j; 

where  Tk  represents  the  effect  due  to  the  generic  factor  under  evaluation  and 
TkTh  represents  the  generic  evaluable  second-order  interaction. 
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Before  starting  the  experiment,  it  is  useful  to  check  if  all  possible  sources  of 
error  variation  have  been  considered,  because  too  high  an  error  variation  can 
compromise  the  ability  to  distinguish  factor  effects.  The  main  error 
variations,  according  to  the  previous  categorization  of  factors,  are 
conceptually  illustrated  in  Exhibit  9.  From  this  description,  we  see  that  the 
preliminary  error  estimation  could  be  underestimated,  since  this  error  did 
not  appear  the  "parameters  setting"  component,  because  all  the  preliminary 
trials  were  made  at  the  same  factor  level.  This  is  evidence  of  how  the  correct 
error  can  be  estimated  only  by  an  analogous  experiment. 

Exhibit  9.  Components  of  measured  experimental  error. 


Experimental  Error 


Misured  Error 


External  Noise 


EEl     =  Erroneous  parameters  setting 

EE2    =  Erroneous  response  measurament 

EE3    =  Erroneous  control  of  steady 
factors 


ENl    =  Variation  due  to  variable  factors 
EN2   =  Variation  due  to  uncontrollable 
factors 

EN3   =  Variation  due  to  unknown  factors 


The  experiment  has  been  conducted  with  fixed  factors  level  and  randomized 
trials  technique,  which  means  that  all  factors  were  kept  at  defined  levels  and 
the  sequence  of  runs  was  random  [Montgomery  D.C.,  1984].  The 
randomization  of  trials  is  an  argued  point,  because,  against  common 
perception,  the  randomization  is  very  expensive  and  the  tendency  to  conduct 
grouped  runs  often  compromises  the  quality  of  the  experiment  [Pignatiello 
J.J.,  1992].  The  main  difficulties  are  due  to  much  setup  and  tuning  to  change 
between  trials.   This  may  significantly  enlarge  experimental  time. 

The  random  sequence  and  outcome  of  the  32  trials  conducted  is  illustrated  in 
Table  3,  where  two  levels  of  factors  are  indicated  with  indexes  1  and  2. 
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Table  3.  Ranciom 

sequence  of  the  trials  anci 

results  of  the  PF]  experiment. 

Trial 

Factors  Combinarion 

tab 

J. Produced       J 
(ml)              ( 

Lost 
ml) 

Yield 

T1(%) 

A 

B              C            Dl           D2 

D3 

1 

2 

2           2           2           2 

2 

32 

51.0 

67.5 

0.430 

2 

2 

2           111 

2 

23 

56.5 

49.5 

0.533 

3 

2 

2          2           11 

2 

24 

61.5 

28.5 

0.683 

4 

1 

112           1 

1 

1 

85.0 

31.0 

0.733 

5 

1 

1112 

1 

9 

118.5 

7.9 

0.938 

6 

2 

1111 

2 

21 

81.0 

58.5 

0.581 

7 

1 

2           12          2 

2 

27 

62.0 

46.5 

0.571 

8 

1 

2           12           1 

1 

3 

89.0 

47.0 

0.654 

9 

2 

12          11 

2 

22 

68.5 

35.5 

0.659 

10 

2 

112           2 

2 

29 

55.0 

62.5 

0.468 

11 

1 

1111 

2 

17 

97.5 

38.0 

0.720 

12 

2 

2           12           2 

2 

31 

47.5 

72.5 

0.396 

13 

2 

2           12           1 

1 

7 

49.5 

76.0 

0.394 

14 

2 

2          2          2           1 

1 

8 

70.0 

57.0 

0.551 

15 

1 

2          2           11 

2 

20 

115.8 

38.5 

0.750 

16 

2 

1           2           1-2 

1 

14 

76.5 

50.0 

0.605 

17 

1 

2           2          2           2 

2 

28 

86.5 

53.5 

0.618 

18 

1 

2           2           2           1 

1 

4 

85.7 

46.2 

0.650 

19 

1 

12          2           1 

1 

2 

90.0 

21.2 

0.809 

20 

2 

12          2          1 

1 

6 

66.5 

41.0 

0.619 

21 

1 

2          111 

2 

19 

80.5 

27.0 

0.749 

22 

1 

12          11 

2 

18 

91.7 

46.0 

0.666 

23 

1 

2          112 

1 

11 

86.0 

20.7 

0.806 

24 

2 

12          2          2 

2 

30 

48.5 

73.5 

0.398 

25 

2 

2          112 

1 

15 

67.5 

50.5 

0.572 

26 

1 

112          2 

2 

25 

73.5 

59.5 

0.553 

27 

1 

12          2          2 

2 

26 

93.5 

39.0 

0.706 

28 

1 

12           12 

1 

10 

110.4 

25.0 

0.815 

29 

2 

2          2           12 

1 

16 

67.7 

32.0 

0.679 

30 

1 

2          2           12 

1 

12 

110.8 

10.8 

0.911 

31 

2 

1112 

1 

13 

77.0 

35.5 

0.684 

32 

2 

112           1 

1 

5 

51.0 

64.0 

0.443 

tab  =  is  the  sequential  order  of  the  combination  in  the  original  matrix. 

To  avoid  typical  forms  of  biasing  in  the  experiment,  it  is  useful  to  follow 
some  elementary  rules:  Make  some  preliminary  trials  just  to  get  confidence 
with  the  equipment.  This  reduces  the  effect  of  learning  curve.  Avoid  any 
kind  of  intermediate  analysis  during  the  experiment.  This  involuntarily 
influences  the  experimenter  to  modify  subsequent  runs  in  the  direction  of 
expectations.  Do  not  repeat  a  trial  because  it  yields  an  apparently  strange 
result. 

4  The  Experiment  Analysis 

The  experimental  phase  is  generally  expensive  and  for  this  reason  we  face  in 
firms  the  tendency  to  reduce  it  as  much  as  possible.  However  it  is  easy  to 
encounter  the  same  brief  approach  even  in  the  analysis  phase.  The 
explanation  often  is  that,  even  if  the  analysis  is  relatively  inexpensive,  it 
requires  a  deep  understanding  of  both  engineering  and  statistics  topics 
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[Phadke  M.S.,  1989].    In  spite  of  abundant  available  literature,  this  problem 
still  presents  many  obstacles  to  adoption  in  industry. 


Exhibit  10.  Different  analyses  involved  in  the  approach. 

1                            Analysis                                 | 

Selected  Factors           | 

•  Architecture 

•  Setting 

Kev  I-actors  Identification 

-  Analysis  of  Variance 

-  Percent   Contributiort 

-  Error  Pooling 

-  Multiple  Comparisons 

-  Regression 

Influenced  Factors     ] 

•  Steady 

•  Minimized 

No  Inference 

It's  not  possible  to  rrmke  inference  on 

the  unchanged  factors. 

Any  architecture  factor  left  out  is  an 

unchanged  factor. 

Free  Factors                      1                                                  1                                                                                                                1 

•  Variable 

•  Uncontrolled 

•  Unknown 

Residual  Analysis 

Negative  inference  from  the  test  of  the 

assumptions. 

Experimental  Error     |                                              |                                                                                                       ] 

•  Set-up 

•  Measurement 

•  Low  control 

Assumptions  Test 

-  Randomness      (Plotting,  Run  Test) 
-Normality             (Shapiro-Wilk  TesO 

-  Independence  oaurbin-Watson  Test) 

-  Homogeneity    (Plotting) 

1                                                      1 

The  main  analyses  to  conduct  are  listed  in  Exhibit  10  and,  although  difficult  to 
separate  sharply,  they  are  organized  in  terms  of  the  factors  they  address  and 
the  types  of  outcomes.  Most  of  the  efforts  are  focused  on  the  identification  of 
key  factors  and  on  the  assumptions  test,  while  remaining  factors  are  only 
indirectly  analyzed  [Anderson  V.L.,  1974;  Hunter  J.S.,  1985]. 

The  first  step  is  the  identification  of  the  key  factors  of  the  phenomenon.  The 
approach  is  based  on  a  preliminary  Analysis  of  Variance  (Anova),  which 
allows  testing  of  the  hypotheses  of  the  model.  We  assume,  for  brevity,  that 
the  reader  has  some  familiarity  with  this  technique,  points  to  the  references 
for  details  [Dunn  O.J.,  1987].  In  some  cases  this  analysis  can  be  conducted  with 
available  software.  In  cases,  like  the  PFJ  application,  due  to  the  complex 
pattern  derived  by  the  nested  factors  and  the  mixed  factorial  structure,  it  is 
necessary  to  formulate  in  an  ad  hoc  fashion  the  expressions  (Appendix  A). 
Table  4a  shows  the  initial  Anova  table  for  the  chosen  model.  Note  that  three- 
factors  and  higher  interactions  are  considered  negligible  and  are  pooled 
together  to  form  the  residual  error  with  9  degrees  of  freedom. 
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Table  4a. 

Initial  Anova  table. 

Source 

dof 

SS 

MS 

F 

p-Value 

n 

A 

1 

0.273 

0.273 

54.944 

0.000 

42.39  % 

B(A) 

2 

0.006 

0.003 

0.626 

0.557 

-0.59  % 

C 

1 

0.018 

0.018 

3.581 

0.091 

2.03  % 

AC 

1 

0.004 

0.004 

0.768 

0.403 

-0.18  % 

BC 

2 

0.008 

0.004 

0.810 

0.475 

-0.30% 

Dl 

1 

0.174 

0.174 

35.009 

0.000 

26.72  % 

D2 

1 

0.000 

0.000 

0.013 

0.912 

-0.78  % 

D3 

1 

0.060 

0.060 

12.073 

0.007 

8.70  % 

ADl 

1 

0.002 

0.002 

0.349 

0.569 

-0.51  % 

AD2 

1 

0.005 

0.005 

1.100 

0.322 

0.08% 

ADS 

1 

0.011 

0.011 

2.144 

0.177 

0.90% 

BDl 

2 

0.010 

0.005 

0.991 

0.408 

-0.01  % 

BD2 

2 

0.000 

0.000 

0.027 

0.973 

-1.53% 

BD3 

2 

0.007 

0.003 

0.692 

0.524 

-0.48  % 

CD! 

1 

0.005 

0.005 

0.912 

0.365 

-0.07  % 

CD2 

1 

0.005 

0.005 

1.040 

0.335 

0.03  % 

CD3 

1 

0.000 

0.000 

0.035 

0.856 

-0.76  % 

Residual 

9 

0.045 

0.005 

24.36  % 

Total 

31 

0.631 

100.00  % 

Table  4b. 

Anova  table  with  error  pooled. 

Source 

dof 

SS 

MS 

F 

p)-Value 

n 

A 

1 

0.273 

0.273 

78.736 

0.000 

42.63  % 

C 

1 

0.018 

0.018 

5.132 

0.004 

2.27  % 

Dl 

1 

0.174 

0.174 

50.169 

0.000 

26.96  % 

D3 

1 

0.060 

0.060 

17.301 

0.000 

8.94  % 

AD2 

1 

0.005 

0.005 

1.576 

0.223 

0.32  % 

AD3 

1 

0.011 

0.011 

3.073 

0.094 

1.14% 

BDl 

2 

0.010 

0.005 

1.444 

0.258 

0.49  % 

CD2 

1 

0.005 

0.005 

1.491 

0.235 

0.27  % 

Residual 

22 

0.076 

0.003 

17.00  % 

Total 

31 

0.631 

100.00  % 

Table  4c. 

Fin?!  Anova  t^ble  w 

th  error  pooled. 

Source 

dof 

SS 

MS 

F 

p-Value 

n 

A 

0.273 

0.273 

73.229 

0.000 

42.59  % 

C 

0.018 

0.018 

4.773 

0.038 

2.22% 

Dl 

0.174 

0.174 

46.661 

0.000 

26.92  % 

D3 

0.060 

0.060 

16.091 

0.000 

8.90  % 

AD3 

0.011 

0.011 

2.858 

0.103 

1.10% 

Residual 

26 

0.097 

0.004 

18.28  % 

Total 

31 

0.631 

100.00  % 

1 

SS  =  Sum  of  Squares,  MS  =  Mean  Square;  F  =  MS/MS  Residual;  FI  =  Percent  Contribution. 
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Also  supplied  in  the  table  is  the  p-Value,  known  as  "probability  value", 
which  gives  the  smallest  significance  level  at  which  the  null  hypothesis  can 
be  rejected.  The  smaller  the  p-value,  the  higher  the  chances  that  a  factor  or 
interaction  is  significant.  Since  high  significance  of  a  factor  or  interaction 
does  not  mean  necessarily  a  strong  impact  on  the  final  performance  of  the 
product,  we  also  provide  the  percent  contribution  of  each  effect,  which  gives 
the  percentage  of  total  variation  due  to  a  factor  or  interaction  (Appendix  B). 
This  information  is  often  important;  in  fact,  as  we  note  in  Table  4  for  the 
head-rotation  factor,  the  significant  level  obtained  (p<0.05)  is  related  to  a  low 
impact  on  performance  variation  Ur=2%. 

Pooling  Technique 

Although  this  is  a  controversial  step  [Davies  O.L.,  1978],  from  Table  4a  we  can 
improve  the  analysis  by  observing  that  the  power  and  the  time  with  many 
other  interactions  have  a  high  p-Value  (p>0.5),  and  so  their  variation  can  be 
considered  as  a  component  of  unexplained  (casual)  variation  and  pooled  in 
the  residual.  Thus  the  residual  reaches  22  degrees  of  freedom,  which  means 
more  accuracy  in  the  error  estimation.  We  can  also  test  better  the  remaining 
factors  and  interaction.  Table  4b  shows  that  the  only  interaction  with  a  low 
p-Value  (p<0.1)  is  the  design-slope  interaction,  so  we  keep  pooling  the  other 
interactions  to  obtain  the  final  model  shown  in  Table  4c  [Ross  P.J.,  1988] 


Table  5.   Anova  table  for  the  final  model. 

Source 

dof 

SS 

MS 

F 

p-Value 

n 

EXPLAINED 

9 

0.555 

0.062 

17.818 

0.000 

83.00  % 

Factors 

4 

0.524 

0.131 

37.834 

0.000 

80.79  % 

Interactions 

5 

0.031 

0.006 

1.806 

0.186 

2.21  % 

RESIDUAL 

22 

0.076 

0.003 

17.00  % 

TOTAL 

31 

0.631 

100.00  % 

MAIN  EFFECTS 

5 

0,535 

0.1069 

28.722 

0.000 

81.72  % 

RESIDUAL 

26 

0.097 

0.0037 

18.28  % 

TOTAL 

31 

0.631 

100.00  % 

1 

Contribution  Analysis 

The  subsequent  step  is  the  Model  Test  and  Percent  Contribution  analysis.    In 

Table  5  we  see  the  synthesis  of  the  model  analysis  in  terms  of  explained 
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factors,  interactions  and  main  effects. 

From  this  information,  beyond  the  significance  of  the  model,  we  can 
determine  the  quality  of  the  approach  by  observing  the  overall  Percent 
Contribution  (n)  of  the  model.  Exhibit  11  shows  the  generic  alternatives  we 
can  encounter.  The  bounds  are  arbitrary  and  only  indicate  a  tendency  rather 
then  a  rule.  In  the  PFJ  case,  the  83%  of  contribution  of  the  final  model  is 
sufficiently  high,  but  still  requires  a  careful  analysis  of  the  residual  to  explain 
eventual  anomalies  either  in  the  framework  or  in  the  pattern  of  experiment. 

Exhibit  11.   Alternatives  in  the  model  contribution  analysis. 


We  can  be  in 
the  known 
domain,  but 


inadequate. 


We  don't  have 
the  right  data, 
which  can  be 
in  the 
unknown 
domain. 


The  explained 
part  is  relevant 
but  it  is 
necessary  to 
perforin  a 
Residual 
Analysis  to 
identify  eventual 
n:\issed  sources 
of  variation. 


The  model 
describe 
adequately  the 
phenomenon, 
v\fe  ca  n  procede 
with  the 
analysis. 


n  =  Percentage  of  the  total  variation  explained  by  the  model. 

In  the  cases  where  the  model  contribution  appears  to  be  very  low,  we  face  the 
situation  where  it  is  likely  necessary  to  change  the  framework.  A  row 
diagnosis  of  whether  we  have  the  right  data  or  not  can  be  made  through  the 
observation  of  the  ratio  between  the  dof  available  for  the  error  and  the  total 
number  of  dof.  When  the  ratio  is  high,  we  should  check  if  everything  in  the 
analysis  was  conducted  correctly  and  if  one  of  the  "not"  selected  factors  is  in 
fact  relevant.  The  alternatives  is  that  missed  key  factors  are  in  the  unknown 
domain  or  the  key  factors  have  been  neutralized  as  steady  factors.  This  is  a 
pertinent  point  in  any  case,  because  during  the  definition  of  the  framework, 
any  architecture  factor  left  out  of  the  selected  set  behaves  as  a  steady  factor, 
and  then  it  will  not  be  possible  to  make  any  inference  on  it.     In  the  PFJ 
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example,  the  head  dimension  was  not  included  in  the  experiment,  and  so  we 
did  not  know  if  it  was  a  key  factor.  At  the  same  time,  we  do  not  know  if  the 
performance  has  been  significantly  affected  by  the  ratio  of  fruit  and  head 
diameter. 

Residual  Analysis 

This  part,  often  neglected,  is  essential  to  complete  the  preceding  analyses  and, 

most  important,  to  verify  if  all  assumptions  underlying  the  methodology  are 

met. 

The  analysis  of  residuals  requires  a  previous  regression  analysis  (Appendix 
C),  from  which  we  calculate  fitted  values  and  residuals.  The  assumptions 
verification  and  residual  interpretation  are  two  sides  of  the  same  coin.  In  fact, 
if  we  detect  any  pattern  in  the  residual  it  means  that  some  assumptions  are 
not  completely  verified  and  therefore  an  unexpected  effect  can  be  in  the 
residual,  and  vice  versa  [Montgomery  D.C.,  1984]. 

The  assumptions  of  the  analysis  of  variance  are:  normality,  independence 
and  homogeneity  of  residuals.  There  are  many  tests  available  in  the 
literature  and  in  some  software  to  verify  the  normality  and  the  independence 
of  residuals,  such  as  the  Shapiro-Wilk  test  and  the  Durbin-Watson  test.  The 
last  assumption  is  the  most  important,  since  the  methodology  is  quite  robust 
to  the  others  [Dunn  O.J.,  1987].  A  complementary  and  easy  approach  to 
accomplish  this  analysis  is  to  plot  the  residuals  against  any  of  the  factors  and 
parameters  that  can  be  considered  worrisome.  In  the  PFJ  residual  analysis  all 
assumptions  were  verified  and  in  spite  of  the  indication  of  model 
contribution  it  was  impossible  to  identify  any  other  form  of  unexpected 
variation.  The  obtained  error  variation  was  explained  with  the  low  precision 
of  the  experiment  (Appendix  D). 

Sensitivity  of  Experiment 

After  the  experimental  phase,  although  defined  in  the  pattern  of  experiment, 
it  is  useful  to  recalculate  the  precision  of  the  experiment,  now  called 
"sensitivity",  with  the  formula  coming  from  the  Fisher's  Least-Significance 
Difference.  This  also  gives  the  confidence  interval  for  the  difference  of  the 
means  estimated  [Mason  R.L.,  1989]: 
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Sen  =  ±  t  [a/2,  V]  (2/n)l/2  Sg 

this  is  relative  to  samples  of  same  size,  and  t[a/2,  v)  is  the  t-distribution  with  an 
upper-tail  of  probability  a/2;  v  =  degree  of  freedom  of  error;  n  =  size  of  each 
sample  compared;  Se  =  error  standard  deviation.  In  the  PFJ  example,  where  a 
=  0.05,  V  =  26,  n  =  16  and  Se  =  0.0608  we  have: 

Sen  =  ±  2.056  *  0.354  *  0.0608  =  ±  4.42  % 

This  indicates  again  the  capacity  of  the  experiment  to  detect  a  variation  in  the 
response  factor  average  of  ±  4.42  %,  which  is  greater  than  the  precision 
previously  imposed.  This  increment  is  essentially  due  to  preliminary 
underestimation  of  the  error  standard  deviation.  Using  the  obtained  Se,  to 
reach  the  initial  precision,  in  the  same  condition,  we  should  perform  at  least 
N=2n=128  trials  (0=0.04/0.0608=0.658).  This  number  would  be  reducible  only 
at  the  cost  of  some  power  of  the  test. 

Multiple  Comparisons 

Another  useful  technique  to  test  the  effects  of  factors  and  interactions 
included  in  the  framework  is  the  multiple  comparisons  test.  This  has  the 
advantage  to  be  easy  to  calculate  and  consents  to  introduce  the 
Experimentalwise  (E)  error,  which  is  the  probability  of  wrongly  rejecting  at 
least  one  null  hypotheses  when  making  statistical  tests  of  two  or  more  null 
hypotheses  using  the  data  from  a  single  experiment  [Mason  R.L.,  1989].  The 
relationship  between  the  experimentalwise  error  and  the  comparisonwise  (a) 
error  defined  previously  is: 

E  =  1  -  ( 1  -  a  )  •^-1 

where  k  is  the  number  of  population  means  compared.  From  this  expression, 
it  is  clear  that,  even  with  a  low  comparisonwise  error  (i.e.  a=0.05),  we  can 
reach  a  high  experimentalwise  error  if  the  number  of  comparisons  increases. 
The  comparison  among  the  four  averages  of  the  head  shape-slope  interaction 
is  reported  in  the  PFJ  example  (Exhibit  12).  The  Bonferroni  procedure  was 
used  to  select  the  comparisonwise  error  (a /2k),  which  is  resulted  to  be 
approximately  0.005  [Ibid.]. 
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Exhibit  12.  Multiple  comparison  diagram  for  the  head  shape-slope  interaction 
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The  Least-Significant-Interval  plotted  using  the  Bonferroni  procedure  is 
LSI={t[a/2k,  v]  (2/n)i/2  Se}/2=  0.0377.  From  the  diagram  it  is  possible  to 
observe  the  effect  of  interaction  as  difference  of  the  slopes  of  the  two  segment 
linking  the  averages.  Also  there  is  an  evident  partial  overlapping  of 
intervals,  which  indicate  the  difficulties  to  distinguish  clearly  the  effects.  This 
is  due  to  more  strict  criterion  adopted  by  this  method  (Appendix  E). 

5  Closing  the  Optimization  Design  Loop 

The  most  impressive  result  was  the  impact  of  head-design  on  the  final 
performance.  The  head  was  responsible  for  more  than  40%  of  the  total 
variation.  In  addition,  the  better  head  design  (type  II)  reduces  the  interaction 
effect  with  slope  factor,  making  the  product  even  more  robust  for  the  user 
(Exhibit  12).  While  the  technology  factor,  the  head  rotation,  -  more  complex 
to  embody  in  the  product-  affected  only  2%  of  the  observed  variation. 

The  experiment  created  new  knowledge  that  went  to  increase  the  previous 
level.  To  illustrate  this  improvement  it  is  useful  to  compare  the  interactions 
table  before  and  after  the  experiment  (Exhibit  13).  In  this  exhibit  we 
distinguish  immediately  the  update  knowledge  and,  more  important,  we  can 
understand  better  the  phenomenon  by  trying  to  explain  the  difference 
between  our  results  and  our  expectations. 
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The  PFJ  experiment  revealed  that  the  time  factor  was  not  significant,  and 
since  it  is  intuitive  that  between  a  very  short  time  (1  sec)  and  a  longer  time  (10 
sec)  there  must  be  a  difference,  we  have  a  row  deduction  of  the  curve 
relationship  between  response  and  the  time. 

Exhibit  13.  Comparison  of  interactions  tables  before  and  after  the  experiment. 


I  Rotation! 


Rotation! 


:  Factor  Nested 
I  Not  Evaluable 


O  =  Strong  Effect 
O  =  Weak  Effect 


From  this  curve  (Exhibit  14)  we  observe  that  in  reality  the  range  chosen 
(R=46%)  was  not  big  enough  to  detect  a  change,  but  from  a  practical  point  of 
view  this  implies  that  excluding  short  uses  of  the  product  the  performance  is 
quite  robust  with  respect  to  time.  Analogous  inferences  can  be  made  for  the 
power  factor. 


Exhibit  14.  Relationship  between  Time  and  Response  factor 
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The  final,  but  no  less  important,  information  created  is  a  better  estimation  of 
the  error  standard  deviation.  This  allows  more  precise  planning  of  any 
subsequent  experiments. 

6  Conclusion 

Although  the  concept  of  quality  keeps  evolving  [Gehani  R.R.,  1993]  and  many 
quantitative  techniques  have  been  developed  for  optimizing  products,  little 
emphasis  has  been  placed  on  identifying  all  their  possible  applications. 

This  work  is  a  further  attempt  to  move  quality  by  design  upstream,  by 
introducing  in  the  experimental  phases  factors  typically  neglected  and 
otherwise  not  easily  evaluable.  Nevertheless,  in  the  improving-through- 
learning  process,  it  is  necessary  to  place  the  proper  emphasis  on  statistical 
competencies,  which  have  to  complement  engineering  skills. 
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Appendix  A 

Siuns  of  Squares  Calculation 

The  formulas  used  in  the  calculation  of  the  sums  of  squares  are  reported  to 
show  the  different  approach  necessary  with  factor  B  nested  in  factor,  A  and 
the  mixed  model  of  Full  Factorial  for  Factors  A,  B,  C  and  Half  Fractional  for 
factors  Dl,  D2,  D3,  which  here  implies  that  the  interactions  DiDj  (for  every  ij) 
are  not  evaluable. 

Defining  the  generic  response  observation  as  Yij^imn/  with  indexes 
corresponding  to  factors  A,  B,  C,  Dl,  D2,  D3  and  varying  between  1  and  2,  we 
start  the  first  step  with  calculation  of  the  Correction  Factor  (CF)  [Montgomery 
D.C.,  1984]: 

CF  =  Y2 /25 

where  dots  indicate  that  indexes  have  been  totalized.    Then  the  Total  sum  of 
squares  can  be  expressed  as: 

SStot  =  ^i  ^j  ^k  ^1  ^m  ^n  Y^ijj^imri   -  CF 
and  all  factors  and  interactions  sums  of  squares  are: 

SSA  =  IiY2i /24  -  CF 

SSb(A)  =  ^i  ^]  Y2ij..../23  -  Ei  Y2i /24 

SSc  =  IkY2..k.../24  -  CF 

SSAC  =  ^A^h-k-n^  -  CF  -  SSa  -  SSc 

SScB(A)  =  5:i5:j5:kY2ijk.../22  -  Zi2:jY2ij..../23  -  EiIkY2i.k.../23  +  IiY2i /24 

SSdi=I:iY2...,../24  -  CF 
SSD2  =  ^mY2....n,./24  -  CF 

SSD3  =  5:nY2 n/24   -  CF 
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SSadi  =2:iSiY2i..,../23  -  CF  -  SSa  -  SSdi 
SSAD2  =  5:i5:mY2i...n,./23   -  CF  -  SSa  "  SSd2 
SSAD3  =  ^i5:nY2i....n/23   -  CF  -  SSa  "  SSd3 

SSb(A)D1  =Si5:jIiY2ij.i../22   -  ZiIjY2ij..../23  -  IiE,Y2i..i../23  -  ZiY2i /24 

SSb(A)D2  =  Si  ^  Sm  Y2ij..n,./22   -   Zj  Zj  Y2ij..../23  "  Ij  Zm  ^h-mJi'  ^  ^  Y^i /2* 

SSb(A)D3  =  li  Sj  In  Y2ij...n/22   -  Ij  Zj  Y2ij..../23  -  Zj  Zn  Y2i....n/23  +  Zi  Y2i /24 

SScDi  =5:k5:iY2..ki../23  -  CF  -  SSc  -  SSdi 
SSCD2  =  ZkZm  Y2..k.m./23  -  CF  -  SSc  -  SSd2 
SScD3  =  SkInY2..k..n/23  -  CF  -  SSc  "  SSd3 

The  sum  of  squares  for  the  residual  can  be  calculated  as  difference  between 
the  total  sum  of  squares  and  the  sum  of  all  sums  of  squares  previously 
calculated: 

SSResidual  =  SStot  -  SSa  "  SSb(A)  "  SSc  -  SSdi  -  SSd2  -  SSd3  -  SSac  -  SSab(A) 

-  SSaDI  -   SSaD2  -   SSaD3  -   SSb(A)D1  -   SSb(A)D2  -   SSb(A)D3  - 

-  SSCDI  -   SScD2  -   SScD3 

It  could  be  easily  demonstrated,  developing  the  formulas  above  shown,  that 
the  effect  of  the  nested  factor  B  is  confounded  with  the  interaction  between 
the  factor  A  and  factor  B  [Montgomery  D.C.,  1984]: 

SSb(A)  =  SSb  +  SSab 

and  analogous  considerations  can  be  made  for  any  interaction  with  the  nested 
factor  B,  for  example: 

SSb(A)C  =  SSbc  +  SSabC 
and  so  on. 

The  value  of  the  correction  factor  was  CF=  12.934  and  in  Table  Al  are  reported 
the  values  assumed  by  the  variables  appearing  in  the  previous  sums,  in 
function  of  their  levels. 
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Table  Al.  Sums  of  Experimental  Observations 


[1] 

[2] 

Yi 

11.649 

8.695 

Y.j.... 

nested 

nested 

Y..k... 

9.795 

10.549 

Y...1.. 

11.351 

8.993 

Y....m. 

10.194 

10.150 

Y n 

10.864 

9.480 

[111 

[12] 

[21] 

[22] 

Yij....* 

5.939 

5.710 

4.456 

4.239 

Y..kl.. 

5.582 

4.213 

5.769 

4.780 

Y..k.m. 

4.807 

4.988 

5.387 

5.162 

Y..k..n 

5.225 

4.570 

5.639 

4.910 

Yi.k... 

5.723 

5.926 

4.072 

4.623 

Yi..l.. 

6.355 

5.294 

4.996 

3.700 

Yi...m. 

5.731 

5.918 

4.463 

4.232 

Yi....n 

6.316 

5.332 

4.548 

4.147 

[111] 

[112] 

[121] 

[122] 

Yijk... 

2.942 

2.996 

2.781 

2.929 

Yij.l.. 

3.138 

2.800 

3.217 

2.493 

Yij..m. 

2.928 

3.011 

2.803 

2.906 

Yij...n 

3.295 

2.644 

3.021 

2.689 

[211] 

[212] 

[221] 

[222] 

Yijk... 

2.177 

2.280 

1.895 

2.344 

Yij.l.. 

2.528 

1.928 

2.467 

1.772 

Yij..m. 

2.301 

2.155 

2.162 

2.077 

Yij...n 

2.351 

2.105 

2.197 

2.043 

The  numbers  on  the  top  rows  indicate  the  values  assumed  by  the  indexes;  for  example,  [211] 
means  i=2,  j=l  and  k=l. 
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Appendix  B 

Percent  Contribution  Calculation 

The  variance  calculated  in  the  fixed  levels  experiment  for  a  factor  or 
interaction,  listed  in  Anova  Table,  contains  some  amount  of  variation  due  to 
the  error  [Phadke  M.S,  1983;  Ross  P.J.,  1988].  The  generic  factor  variance 
observed  (Vx  )  can  be  written  as: 

Vx   =  V*x  +  Ve 

where  V*x  is  the  variance  due  solely  to  the  factor  x  and  Vg  is  the  variance  of 
the  error.   The  pure  variation  of  factor  x  can  be  isolated  as: 

V*x  =  Vx  -  Ve 

and  since  in  the  Anova  Table  the  variance  is  expressed  as  the  ratio  of  sum  of 
squares  and  degree  of  freedom  of  factor  (Vx=SSx/Vx)  we  have: 

SS*x/Vx  =  SSx/Vx-Ve 
Solving  for  SS*x  we  obtain: 

SS*x  =  SSx  -  Ve  X  Vx 

then  the  percent  contribution  of  the  factor  x  (Fix)  respect  the  total  variation 
expressed  in  terms  of  sum  of  squares  can  be  calculated  as: 

Ox  =  SS\  /  SSTot  X  100 

The  contribution  of  error  (residual)  then  is  calculated  replacing  on  it  all 
variation  removed  by  the  factors  and  interactions: 

He  =   (SSx  +  dof  F.l.  X  Ve)  /  SSjot  x  100 

where  dof  p.i.  is  the  total  number  of  degree  of  freedom  available  for  factors  and 

interactions. 

This  correction  is  necessary  to  avoid  overestimating  the  contribution  of  the 

effects,  and  it  is  cause  of  the  presence  of  negative  number  in  the  initial  Anova 

Table  (14a). 
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Appendix  C 


Regression  Analysis 


The  regression  analysis,  conducted  with  the  MINITAB®  software  ,  confirms  the 
results  obtained  with  the  Anova  Analysis  for  the  factors  and  for  the  model, 
then  it  produces  residual  and  fitted  values  necessary  for  the  Residual 
Analysis,  the  regression  results  are  listed  in  Table  CI: 


Table  Cl.  Regression 

Table 

Predictor 

Coef. 

St.,Dev. 

t-ratio 

p-Value 

VIF 

Constant 

0.840 

0.026 

31.760 

0.000 

A 

-0.221 

0.031 

-7.240 

0.000 

2 

C 

0.047 

0.022 

2.180 

0.038 

1 

D1 

-0.147 

0.022 

-6.830 

0.000 

1 

D3 

-0.123 

0.031 

-4.020 

0.000 

2 

A*D3 

0.073 

0.043 

1.690 

0.103 

3 

Source 

dof 

SS 

MS 

F 

p-Value 

Regression 

5 

0.535 

0.1069 

28.67 

0.000 

Error 

26 

0.097 

0.0037 

Total 

31 

0.631 

The  regression  equation  using  the  coefficients  shown  in  Table  Cl  is: 

Tj  =  0.840  -  0.221  A  +  0.047  C  -  0.147  Dl  -  0.123  D3  -i-  0.073  AD3 

The  typical  indicators  of  the  goodness  of  the  model,  as  the  R2  and  the  R^ady 
are  below  reported  : 

R2=84.6%       and       R2adj  =  81.7  % 

The  software  also  provides  other  information  as:  the  absence  of  evidence  of 
Lack  Of  Fit  (p  >  0.1)  and  the  pure  error  test: 

F  =  MSLOF/MSpc  =1.27    (p- Value  =  0.325  } 
where  the  pure  error  (pe)  has  16  degree  of  freedom. 
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The  residuals  obtained  from  the  regression  analysis  are  listed  in  Table  C2, 
where  it  is  also  shown  fitted  values,  sorted  residuals  and  normal  quantile 
values. 


Table  C2.  Residuals  of  the  final  models. 


Trials 

Residuals 

Fits 

Sort.  Res. 

n-quantil 

1 

-0.038 

0.468 

-0.098 

-2.070 

2 

-0.036 

0.569 

-0.089 

-1  .643 

3 

0.067 

0.616 

-0.077 

-1  .396 

4 

0.041 

0.692 

-0.072 

-1  .213 

5 

0.098 

0.840 

-0.070 

-1.064 

6 

0.012 

0.'569 

-0.061 

-0.935 

7 

0.002 

0.569 

-0.047 

-0.820 

8 

-0.038 

0.692 

-0.038 

-0.716 

9 

0.043 

0.616 

-0.038 

-0.618 

1  0 

0.047 

0.421 

-0.036 

-0.527 

1  1 

0.003 

0.717 

-0.034 

-0.440 

1  2 

-0.025 

0.421 

-0.028 

-0.356 

1  3 

-0.077 

0.471 

-0.025 

-0.274 

1  4 

0.033 

0.518 

-0.016 

-0.195 

1  5 

-0.014 

0.764 

-0.014 

-0.1  16 

1  6 

-0.061 

0.666 

0.002 

-0.039 

1  7 

0.002 

0.617 

0.002 

0.039 

1  8 

-0.089 

0.739 

0.003 

0.116 

1  9 

0.070 

0.739 

0.012 

0.195 

20 

0.101 

0.518 

0.013 

0.274 

21 

0.032 

0.717 

0.024 

0.356 

22 

-0.098 

0.764 

0.032 

0.440 

23 

-0.034 

0.840 

0.033 

0.527 

24 

-0.070 

0.468 

0.041 

0.618 

25 

-0.047 

0.619 

0.043 

0.716 

26 

-0.016 

0.569 

0.047 

0.820 

27 

0.090 

0.617 

0.066 

0.935 

28 

-0.072 

0.887 

0.067 

1.064 

29 

0.013 

0.666 

0.070 

1.213 

30 

0.024 

0.887 

0.090 

1.396 

31 

0.066 

0.619 

0.098 

1.643 

32 

-0.028 

0.471 

0.101 

2.070 

32 


Appendix  D 

Residuals  Analysis 

Using  the  regression  results,  it  is  possible  to  perform  the  residuals  analysis 
with  the  intent  to  test  the  basic  assumptions  of  the  Anova  Analysis  and  to 
investigate  if  some  unpredicted  sources  of  variation  are  detected. 
The  assumptions  to  test  are:  independence,  normality  distribution  and 
homogeneity  of  errors.  The  first,  and  most  important,  can  be  tested  with  the 
Durbin-Watson  method,  which  requires  the  calculation  of  the  statistic: 

d  =  Zi  (fi  -  ri.i)2  /  ZiT^i  with  i  =  2, 3,...32 

where  T\  is  the  residual,  and  the  value  d*=1.80  obtained  must  be  compared 
with  the  bounds  du  and  dt  (1.18,  1.73  for  a=0.05)  from  the  tables  available  in 
literature  [Mason  R.L.,  et  al.,  1989].  To  test  the  hypothesis  Hq:  errors  not 
autocorrelated  vs  Hg:  errors  positively  autocorrelated,  we  can:  I)  reject  Hq  if  d< 
dt,  11)  do  not  reject  Hq  if  d>du;  HI)  draw  no  conclusion  if  dL<d<du.  Analogous 
reasoning  is  required  for  negative  autocorrelation.  In  the  FPS  case  d*>du  and 
we  did  not  reject  Hq,  which  means  that  the  independence  assumption  was 
supported. 

The  normality  assumption  can  be  tested  using  the  Shapiro-Wilk  method, 
which  requires  to  order  ascending  the  residuals  and  calculate  the  statistic: 

W  =  b2  /  S2 

where  b  =  Zi  ai  r^j  and  S^  =  Ei  (ri.Ijri/n  )2  with  i=l,2,...,n  and  from  the 
tables  [ibid.]  we  obtain  the  values  aj  and  the  critical  value  for  W.  The 
hypothesis  of  normality  is  rejected  if  the  value  W  obtained  is  less  then  W- 
critical.  The  value  for  the  FPS  case  was  W*=0.966,  that  compared  with  the 
W(n=32,  a=o.45)=  0.965  did  not  allow  us  to  reject  the  hypothesis  of  normality. 
Independence  and  homogeneity  assumptions  can  be  further  investigated 
plotting  the  residuals  vs  the  factors  and  interactions  (exhibit  Dl),  vs  fitted 
values  and  time  (exhibit  D2,  D3);  the  normality  assumption  with  the  plot  of 
the  residuals  vs  the  fitted  values  and  the  normal  quantile  values  (exhibit  D3, 
D4).  These  do  not  show  any  pattern  to  worry  about.  On  the  other  hand,  they 
do  not  help  in  finding  other  missed  sources  of  variation. 
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Table  Dl. 

Residuals  vs  Main  Factors. 
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Exhibit  D2.  Plot  Residuals  vs  Experimental  Time 
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Exhibit  D3.   Plot  Residuals  vs  Fitted  Values. 
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Exhibit  D4.   Plot  Residuals  vs  Normal-Quantile. 
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Appendix  E 


Exhibit  El. 

Multiple  ComparisonsPlots:  response  factor  vs  main  factors. 
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Appendix  F 

Table  Fl.  Sample  size  requirements  for  tests  on  difference  of  two  means  from  independent 
normal  population,  equal  population  standard  deviations. 


Level 

of  t-Test 

single-side 

0=0.005 

0=0.01 

0=0.025 

0=0.05 

double-side 

0=0.01 

0=0.02 

0=0.05 

0=0.1 

P  = 

0.01 

0.05 

0.1 

0.2 

0.01 

0.05 

0.1 

0.2 

0.01 

0.05 

0.1 

0.2 

0.01 

0.05 

0.1 

0.2 

0.30 

0.35 

102 

0.40 

100 

108 

78 

0.45 

118 

101 

105 

79 

108 

86 

62 

0.50 
0.55 

96 

106 

82 

106 

86 

64 

88 

70 

51 

101 

79 

106 

88 

68 

87 

71 

53 

112 

73 

58 

42 

0.60 

101 

85 

67 

-90 

74 

58 

104 

74 

60 

45 

89 

61 

49 

36 

0.65 

87 

73 

57 

104 

77 

64 

49 

88 

63 

51 

39 

76 

52 

42 

30 

0.70 

100 

75 

63 

50 

90 

66 

55 

43 

76 

55 

44 

34 

66 

45 

36 

26 

0.75 
0.80 

88 

66 

55 

44 

79 

58 

48 

38 

67 

48 

39 

29 

57 

40 

32 

23 

11 

58 

49 

39 

70 

51 

43 

33 

59 

42 

34 

26 

50 

35 

28 

21 

0.85 

69 

51 

43 

35 

62 

46 

38 

30 

52 

37 

31 

23 

45 

31 

25 

18 

0.90 

62 

46 

39 

31 

55 

41 

34 

27 

47 

34 

27 

21 

40 

28 

22 

16 

0.95 

55 

42 

35 

28 

50 

37 

31 

24 

42 

30 

25 

19 

36 

25 

20 

15 

1.00 
1.10 

50 

38 

32 

26 

45 

33 

28 

22 

38 

27 

23 

17 

33 

23 

18 

14 

42 

32 

27 

22 

38 

28 

23 

19 

32 

23 

19 

14 

27 

19 

15 

12 

1.20 

36 

27 

23 

18 

32 

24 

20 

16 

27 

20 

16 

12 

23 

16 

13 

10 

<|)=|5|/o  1.30 

31 

23 

20 

16 

28 

21 

17 

14 

23 

17 

14 

11 

20 

14 

11 

9 

1.40 

27 

20 

17 

14 

24 

18 

15 

12 

20 

15 

12 

10 

17 

12 

10 

8 

1.50 
1.60 

24 

18 

15 

13 

21 

16 

14 

11 

18 

13 

11 

9 

15 

11 

9 

7 

21 

16 

14 

11 

19 

14 

12 

10 

16 

12 

10 

8 

14 

10 

8 

6 

1.70 

19 

15 

13 

10 

17 

13 

11 

9 

14 

11 

9 

7 

12 

9 

7 

6 

1.80 

17 

13 

11 

10 

15 

12 

10 

8 

13 

10 

8 

6 

11 

8 

7 

5 

1.90 

16 

12 

11 

9 

14 

n 

9 

8 

12 

9 

7 

6 

10 

7 

6 

5 

2.00 
2.10 

14 

11 

10 

8 

13 

10 

9 

7 

n 

8 

7 

6 

9 

7 

6 

4 

13 

10 

9 

8 

12 

9 

8 

7 

10 

8 

6 

5 

8 

6 

5 

4 

2.20 

12 

10 

8 

7 

11 

9 

7 

6 

9 

7 

6 

5 

8 

6 

5 

4 

2.30 

11 

9 

8 

7 

10 

8 

7 

6 

9 

7 

6 

5 

7 

5 

5 

4 

2.40 

11 

9 

8 

6 

10 

8 

7 

6 

8 

6 

5 

4 

7 

5 

4 

4 

2.50 
3.00 

10 

8 

7 

6 

9 

7 

6 

5 

8 

6 

5 

4 

6 

5 

4 

3 

8 

6 

6 

5 

7 

6 

5 

4 

6 

5 

4 

4 

5 

4 

3 

3.50 

6 

5 

5 

4 

6 

5 

4 

4 

5 

4 

4 

3 

4 

3 

4.00 

6 

5 

4 

4 

5 

4 

4 

3 

4 

4 

3 

4 

Note:  The  entries  in  this  table  show  the  number  of  observations  needed  (for  each  of  two 
samples  of  equal  size)  in  a  test  of  significance  of  the  difference  between  two  means  in  order  to 
control  the  probabilities  of  the  errors  of  the  first  and  second  kinds  at  a  and  b  respectively. 
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